vDNN: Virtualized Deep Neural Networks for Scalable, Memory-Efficient Neural Network Design

The most widely used machine learning frameworks require users to carefully tune their memory usage so that the deep neural network (DNN) fits into the DRAM capacity of a GPU. This restriction hampers a researcher's flexibility to study different machine learning algorithms, forcing them to either use a less desirable network architecture or parallelize the processing across multiple GPUs. We propose a runtime memory manager that virtualizes the memory usage of DNNs such that both GPU and CPU memory can simultaneously be utilized for training larger DNNs. Our virtualized DNN (vDNN) reduces the average GPU memory usage of AlexNet by up to 89%, OverFeat by 91%, and GoogLeNet by 95%, a significant reduction in memory requirements of DNNs. Similar experiments on VGG-16, one of the deepest and memory hungry DNNs to date, demonstrate the memory-efficiency of our proposal. vDNN enables VGG-16 with batch size 256 (requiring 28 GB of memory) to be trained on a single NVIDIA Titan X GPU card containing 12 GB of memory, with 18% performance loss compared to a hypothetical, oracular GPU with enough memory to hold the entire DNN.Comment: Published as a conference paper at the 49th IEEE/ACM International Symposium on Microarchitecture (MICRO-49), 201

Optimizing Multi-GPU Parallelization Strategies for Deep Learning Training

Deploying deep learning (DL) models across multiple compute devices to train large and complex models continues to grow in importance because of the demand for faster and more frequent training. Data parallelism (DP) is the most widely used parallelization strategy, but as the number of devices in data parallel training grows, so does the communication overhead between devices. Additionally, a larger aggregate batch size per step leads to statistical efficiency loss, i.e., a larger number of epochs are required to converge to a desired accuracy. These factors affect overall training time and beyond a certain number of devices, the speedup from leveraging DP begins to scale poorly. In addition to DP, each training step can be accelerated by exploiting model parallelism (MP). This work explores hybrid parallelization, where each data parallel worker is comprised of more than one device, across which the model dataflow graph (DFG) is split using MP. We show that at scale, hybrid training will be more effective at minimizing end-to-end training time than exploiting DP alone. We project that for Inception-V3, GNMT, and BigLSTM, the hybrid strategy provides an end-to-end training speedup of at least 26.5%, 8%, and 22% respectively compared to what DP alone can achieve at scale

Genome-wide identification and phenotypic characterization of seizure-associated copy number variations in 741,075 individuals

Copy number variants (CNV) are established risk factors for neurodevelopmental disorders with seizures or epilepsy. With the hypothesis that seizure disorders share genetic risk factors, we pooled CNV data from 10,590 individuals with seizure disorders, 16,109 individuals with clinically validated epilepsy, and 492,324 population controls and identified 25 genome-wide significant loci, 22 of which are novel for seizure disorders, such as deletions at 1p36.33, 1q44, 2p21-p16.3, 3q29, 8p23.3-p23.2, 9p24.3, 10q26.3, 15q11.2, 15q12-q13.1, 16p12.2, 17q21.31, duplications at 2q13, 9q34.3, 16p13.3, 17q12, 19p13.3, 20q13.33, and reciprocal CNVs at 16p11.2, and 22q11.21. Using genetic data from additional 248,751 individuals with 23 neuropsychiatric phenotypes, we explored the pleiotropy of these 25 loci. Finally, in a subset of individuals with epilepsy and detailed clinical data available, we performed phenome-wide association analyses between individual CNVs and clinical annotations categorized through the Human Phenotype Ontology (HPO). For six CNVs, we identified 19 significant associations with specific HPO terms and generated, for all CNVs, phenotype signatures across 17 clinical categories relevant for epileptologists. This is the most comprehensive investigation of CNVs in epilepsy and related seizure disorders, with potential implications for clinical practice

Transient stability enhancement in renewable energy integrated multi-microgrids: A comprehensive and critical analysis

Multi-microgrids offer various benefits including the decreased overloading of a single microgrid, more options during faulty conditions, and more utilization of renewable energy resources. However, the implementation of a multi-microgrid brings the challenges such as power system complexity, interconnection issues, bidirectional power flow management, and power flow balancing. In the presence of these challenges, the power flow stability of the multi-microgrids is a challenging problem. In this context, this study evaluates a transient stability analysis model in multi-microgrids using solar photovoltaics, wind power, and a unified power flow controller (UPFC). UPFC offers a more robust power flow control strategy compared with other flexible alternating current transmission systems (FACTS) devices. First, a multi-microgrid structure consisting of the two microgrids was designed in DIgSILENT PowerFactory software. Second, the load flow calculation was performed in the absence and presence of UPFC, short circuit fault, and grid connection. Third, the electromagnetic transients (EMT) simulation was performed for all these situations. The results exhibited that the UPFC would offer significant power flow stability in the multi-microgrids. It was observed that the UPFC resulted in more transient stability in the microgrid where it was located. However, it improved the power flow quality at all the locations in the multi-microgrids. In addition, UPFC offered significant transient stability during the fault occurrence. The results offer various insights into power flow management in multi-microgrids

Effect of Soil Texture, Nanoparticle Size, and Incubation Period on the Dissolution of ZnO Nanoparticles

Zinc is an essential plant and human nutrient and its primary source is Zn-rich food consumption. The only way to enrich plants with Zn is through the application of Zn fertilizers including various chemical and organic sources of ZnO nanoparticles (NPs). The Zn bioavailability from ZnO NPs must be considered for their recommendation as a fertilizer, and very little is known about the efficacy of such fertilizers in the Hungarian soil environment. In the present investigation, we prepared ZnO NPs of different sizes and applied them in two distinct textures of soils (sandy loam (SL) and silty clay (SC)) in an incubation experiment. The prepared ZnO NPs were characterized using X-ray diffraction (XRD) and scanning electron microscopy (SEM). ZnO NPs were applied in both soil types at 500 mg L−1 in the form of a suspension, and ZnSO4 was applied in the form of a solution. The soils were incubated for 7 and 14 days. Column leaching was performed to analyze the dissolved Zn. Retained Zn in the soil matrix was extracted using 0.05 M EDTA. The results showed that approximately 21–23% and 10–13% higher Zn was observed in the pore water of SL and SC soils, respectively, when spiked with small-sized NPs compared to large-sized NPs, while 14–26% higher dissolved Zn was observed in SL soil compared to SC soil. It is concluded that the size of NPs and the soil texture are the main factors that play important roles in deciding the fate of NPs under an alkaline soil environment

Challenges in organic component selection and biochar as an opportunity in potting substrates: a review

Plant production in potting substrates provides maximum profit on the applied inputs, and hence, directly improving the socio-economic condition of the grower/nurserymen. The main challenge in this industry is sourcing of materials for their potting substrates. Peat and perlite have been widely preferred materials. However, recently higher prices, more restrictive legislation of many countries and wetland ecosystem destruction through its extraction has limited peat use. Nowadays, producers focus towards peat alternatives that provide good performance, are readily available, inexpensive and environment friendly to attain sustainability in potted plant production. In an effort to grasp sustainability during the last few decades, many industrial and agricultural waste materials were reviewed for their use in potting substrates. In these studies, the major focus remained on material characterization, neglecting their economics, technical aspects and environmental impacts. Thus, switching from peat and perlite to alternatives requires material exploration. In the present review, we summarize a clearer and practical approach for substituting different materials especially biochar to fulfill the need of modern potting substrate industry. Biochar has the potential to sustain the substrate production on a long-term basis